Performance testing

There are several options available useful in performance testing.

HTTP fixed delays

Configurable via the UI. Just change the value of "Response delay" on the HTTP mapping form.

HTTP random delays

Configure your mapping file to contain:

...
    "response": {
            "status": 200,
            "delayDistribution": {
                    "type": "lognormal",
                    "median": 70,
                    "sigma": 0.3
            }
...

JMS delays

JMS delays are available, see the documentation for more details.

Native IBM® MQ delays

JMS delays are available, see the documentation for more details.

Performance profile allows for faster than usual responses in Traffic Parrot.

To enable the performance profile:

1. use trafficparrot.performance.properties
2. configure only the extensions and helpers that are in use in the mappings, by default all of them are disabled
3. disable INFO logs (set log4j to WARN or ERROR only)
4. configure the operating system
5. Optional: run a cluster of Traffic Parrot instances behind a load balancer
6. Optional: if you have any issues contact us

Please note that all mappings in performance mode are loaded into memory from the disk on load so you if you make changes directly on disk to any mappings please do not do it in performance mode.

Native IBM MQ performance monitoring

For Native IBM MQ you can enable additional logging that displays total processing time of messages.

Here is a list of configuration parameters for tuning Traffic Parrot Native IBM® MQ.

Traffic Parrot properties:

• trafficparrot.virtualservice.mapping.cache.milliseconds - how long to mapping cache the files in memory (default 0 means no caching)
• trafficparrot.virtualservice.mapping.cache.populate.on.startup - should the mappings cache be pre-loaded on startup

Native IBM® MQ connection attributes:

• readConnectionsToOpen - how many read connections to open to the queue manager via the specified channel
• writeConnectionsToOpen - how many write connections to open to the queue manager via the specified channel

For example:

{
    "connectionId": "1",
    "connectionName": "Local Docker MQ 9",
    "connectionData": {
      "ibmMqVersion": "IBM_MQ_9",
      "hostname": "localhost",
      "port": 1414,
      "queueManager": "QM1",
      "channel": "DEV.APP.SVRCONN",
      "username": "app",
      "password": "",
      "useMQCSPAuthenticationMode": false,
      "readConnectionsToOpen": 5,
      "writeConnectionsToOpen": 5,
      "sslCipherSuite": null,
      "sslPeerName": null
    }
},

Native IBM® MQ attributes:

• receiveThreads - how many threads to use for the given mapping to receive messages
• sendThreads - how many threads to use for the given mapping to send messages

For example:

{
  "mappingId" : "3bc18f0b-9d95-4af1-a2f8-848210b2d8a1",
  "request" : {
    "destination" : {
      "name" : "DEV.QUEUE.1",
      "type" : "QUEUE"
    },
    "bodyMatcher" : {
      "anything" : "anything"
    }
  },
  "response" : {
    "destination" : {
      "name" : "DEV.QUEUE.2",
      "type" : "QUEUE"
    },
    "ibmMqResponseTransformerClassName" : "NO_TRANSFORMER",
    "format" : "MQFMT_STRING",
    "text" : "",
    "fixedDelayMilliseconds" : 0
  },
  "receiveThreads" : 5,
  "sendThreads" : 5
}

If you are observing lower than expected performance please contact us.

Please keep in mind that Traffic Parrot performance depends on:

• speed of hardware or VM, the slower the hardware or VM the slower Traffic Parrot
• Java version, the older the version the lower the performance
• number of mappings, the higher number of mapping the lower the performance
• complexity of dynamic responses, the higher the complexity the lower the performance
• complexity of the request matching, an "any" matcher will be much faster than a complex one like regexp or jsonpath

Summary

This benchmark demonstrates how hardware resources, network parameters and complexity of virtual services (mocks/stubs) impact Traffic Parrot version 5.12.0 performance for a few sample scenarios. In one example, we show how we improve from 6,000 TPS to 20,000 TPS by increasing the hardware resources and network capacity. In another example, we show that the complexity of the virtual services (mocks/stubs) results in the difference between 1,000 TPS and 6,000 TPS when running on the same hardware and network. Download benchmark PDF here of view the results in web browser below.

Benchmark results

Test setup		Test results
		4 vCPUs HDD 6GB heap 10 Gb/s network		16 vCPUs SSD 12GB heap 10 Gb/s network		16 vCPUs SSD 12GB heap 30 Gb/s network
Request to response mappings (transactions defined in the virtual service)	Queues and queue managers	TPS	Processing latency (read request message, construct response message, write response message)	TPS	Processing latency (read request message, construct response message, write response message)	TPS	Processing latency (read request message, construct response message, write response message)
20 XML mappings, 100ms fixed delay, Dynamic (2 XPaths), Message size 490B, 1 send threads per queue, 1 receive threads per queue, 5 read connections per  QM, 5 write connections per QM, Non-transactional, non-persistent	20 queues 4 queue managers	6,022t/s 10,000,000 transactions	99% under 50.00ms 95% under 20.00ms	14,984t/s 10,000,000 test transactions	99% under 40.00ms 95% under 30.00ms	21,541t/s 10,000,000 test transactions	99% under 30.00ms, 95% under 20.00ms
20 XML mappings, No delay, Dynamic (2 XPaths), Message size 490B, 1 send threads per queue, 1 receive threads per queue, 5 read connections per  QM, 5 write connections per QM, Non-transactional, non-persistent	20 queues 4 queue managers	5,751t/s 10,000,000 transactions	99% under 30.00ms 95% under 20.00ms	13,425t/s 10,000,000 test transactions	99% under 50.00ms 95% under 30.00ms	19,321t/s 10,000,000 test transactions	99% under 30.00ms 95% under 20.00ms
15 XML mappings fixed delays 100ms to 200ms Dynamic (1 to 29 XPaths per message) Message size 500B to 57kB 1-4 send threads depending on the queue 1-4 receive threads depending on the queue 18 read connections per QM 18 write connections per QM Non-transactional, non-persistent	15 queues 2 queue managers	1,276t/s 3,080,000 transactions	99% under 10.00ms 95% under 10.00ms	4,180t/s 3,080,000 test transactions	99% under 10.00ms 95% under 10.00ms	4,472t/s 3,080,000 test transactions	99% under 10.00ms 95% under 10.00ms

Environment configuration 4 vCPUs - 6GB heap - 10 GB/s network

Testing Traffic Parrot version 5.12.0-RC1

IBM MQ 9.1.1.0

TP running on

• GCP VM type n2-standard-4 (4 vCPUs, 16 GB memory)
• Standard persistent disk
• “europe-west2-c” GCP data center

TP configuration

• Logging level = WARN
• -Xmx6144m

Network

• Using external IPs (10 Gbits/sec tested with iperf)

Four IBM MQ queue managers each of them running on

• GCP VM type n1-standard-1 (1 vCPU, 3.75 GB memory)
• Standard persistent disk
• “europe-west2-c” GCP data center

Environment configuration 16 vCPUs - 12GB heap - 10 Gb/s network

Testing Traffic Parrot version 5.12.0-RC1

IBM MQ 9.1.1.0

TP running on

• GCP VM type c2-standard-16 (16 vCPUs, 64 GB memory)
• SSD persistent disk
• “us-central1-a” GCP data center

TP configuration:

•  Logging level = WARN
• -Xmx12g

Network

• Using external IPs (10 Gbits/sec tested with iperf)

Four IBM MQ queue managers each of them running on

• GCP VM type n2-highcpu-4 (4 vCPUs, 4 GB memory)
• SSD persistent disk
• “us-central1-a” GCP data center

Environment configuration 16 vCPUs - 12GB heap - 30 Gb/s network

Same as “16 vCPUs - 12GB heap - 10 Gb/s network” above but:

Network

• Using internal IPs (30Gb/s tested with iperf)